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ABSTRACT 


A CAD package for the design of linear quadratic regulators 
using output feedback is developed. Various existing methods 
for the design were studied and an efficient algorithm, incor- 
porating better features of some of them, was developed using 
linear static and dynamic output feedback. Several numerical 
examples have been worked out using the package developed. The 
user’s guide for the use of the package developed is also given. 
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Notations 


1. 1 

C 

C 

^nxm 

R 

P^nxm 
<» > 

ll-llp 

A > 0 
A > 0 
A’ 

Tr[.] 

eC 3 

dj/dA 

/(A) 

X 

Col(A) = 


determinant of a matrix valued function or the 
absolute value of a scalar function 

field of complex numbers 

field of complex numbers with real part < 0 

matrices with n rows and m columns with elements in 

field of real numbers 

matrices with n rows and m columns with elements in 

usual meaning for scalar and element by element 

application to matrices 

p-norm 

positive definite A 
non-negative definite A 
transpose of A 
trace of a matrix 
expected value 

gradient of scalar J with respect to matrix A 
spectrum of A 

a vector consisting of elements Xj^,X 2 » ••• 

~ A 2 * • • • » i$i^3 

Aj^ - ith column of A 



CHAPTER 1 


INTRODUCTION 

The cornerstone of modern control theory judging from 
frequency of applications is the linear constant gain feed- 
back control determined by minimizing a quadratic index of 
performance. With this LQR design approach, which is one of 
the most powerful state space methods of designing multi- 
variable systems, design problems are reduced to choosing 
weighting matrices in the quadratic performance index. More- 
over, the resulting closed loop system has good robustness 
properties. 

But the implementation of LQR regulators require availa- 
bility of all states of the system for measurement and feed- 
back which, however, will be rare in practical situations. 
When the complete set of states are not available, in the 
case of stochastic systems a Kalman filter and in the case 
of deterministic system a Luenberger observer could be used 
to estimate the states and design an optimal controller. But 
the order of Kalman filter is equal to that of the system 
making the use of it for high order systems prohibitive. Al- 
though a full order observer too has an order the same as 
that of the system, reduced order observers and minimal order 
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or function observers exist to estimate the states. But 
even in these cases, the order may be too high for economical 
implementation. 

Industrial processes are generally of high order and 
use of observers and optimal state control will be too sophi- 
sticated when often a simple controller that would stabilize 
the process in some operating region and achieve some desired 
behaviour would suffice. In such cases memoryless or state 
output feedback using only the available measurements will be 
suitable, making the structure of feedback simple. 

The index of performance of optimal output feedback 
regulator is of course worse than that of the standard state 
feedback regulator, but this is the price paid for simplicity. 

In many cases, however, the performance of optimal output 
feedback regulators is close to that of standard regulator, 
thus justifying the use of simpler feedback structures. 

There will be cases when there exists no static output 
feedback which can optimize a performance index. This happens 
when the system is not output stabilizable. Then a low order 
dynamic controllers is designed such that the overall closed 
loop system has satisfactory and desirable response chara- 
cteristics. Almost all systems can be stabilized by dynamic 
output feedback, when they are controllable and observable [11 ]• 
No explicit estimation of states is done in dynamic feedback 
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controllers, which uses only available outputs for feedback. 
Since these methods do not require input sensing, unlike an 
observer, these have wider applications, especially when 
system inputs are unknown, since observers are not unique 
selection of a particular observer is not easy. 

In many applications the structure of feedback will have 
to be constrained, wherein some of the feedback elements are 
zeroes. One such case is that of a decentralized control of 
a geographically distributed system in which exchange of 
information can be prohibitively costly. It further simpli- 
fies the feedback structure and helps in eliminating unpro- 
ductive elements in the feedback structure. 

Unlike finding an optimal state feedback, which involves 
only the solution of an associated algebraic Riccati equation, 
the problem of finding an optimal output feedback is 
nontrivial and encounters many numerical problems. 

The problem has been studied in detail and different 
solutions have come out, considering the cost and availability 
of computing system etc. Finer aspects of the problem like 
minimizing the order of compensator, developing numerically 
stable and efficient algorithms have taken significance. 

In the work presented here, the algorithm mainly relied 
upon is a modified version of Anderson Moore method [42] 
which is based on a set of necessary conditions derived by 
Levine and Athans [39]. 



The thesis is organised as follows ; Chapter 2 gives 
a brief survey of the literature in the area of output feed- 
back regulators; Chapter 3 presents the static output feed- 
back problem, algorithms used and briefly describes the prim- 
ing feedback problem, search direction and unidimensional 
search along this direction; Chapter 4 describes the dynamic 
output feedback problem and how it is transformed into an 
equivalent static output feedback regulator problem; Chapter 6 
analyses and discusses some of the results and Chapter 5 gives 
guidelines to the user for the use of the design package 
developed with the algorithm. Chapter 7 gives the conclusions. 



CHAPTER 2 


LITERATURE SURVEY 

Linear quadratic optimal output feedback control has 
continuously stimulated research in the area of output feed- 
back control, due to simplicity of constant feedback stru- 
cture, ease in extending to dynamic output feedback and 
challenge in numerical solution for computing optimal gains. 

The problem has been considered by several authors, but 
being a nontrivial, computational problem, it has not yet 
found a fully acceptable solution. 

Significant developments in the direction of finding 
optimal control with only accessible states were made in 
the early 1970' s. However, progress in the direction of 
finding efficient algorithms for implementation were made 
much later. One of the problems in the design of optimal 
output feedback was and still is that of the choice of a 
performance index the minimization of which will result in 
a regulator as close in terms of the index as possible to 
the standard optimal regulator. Another problem associated 
with the design is the undesirable dependence of the feed- 
back gain elements on the initial states, Xq. 
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Minimization of maximum of the eigenvalues of the 
associated cost matrix, S, was done by Man [43], He showed 
that this method is suprior, i.e., cost is always smaller, 
than minimizing the norm of the deviation of this matrix 
from the standard optimal cost matrix. An equivalent method 
was presented by Rekasius [53 j in which, maximum, for any 
initial state, of the ratio of suboptimal cost to optimal 
cost was minimized. An upper bound of the above cost, 
Euclidean norm of cost matrix, was minimised in [iS], Levine 
and Athans [39] took the same performance index as in stan- 
dard regulator and obtained necessary conditions for the 
solution. The dependency of the solution on the initial 
state was eliminated by assuming the initial state to be a 
random vector uniformly distributed over the unit hypersphere 
and minimizing the expected value of the cost. A general . 
performance index, (1), based on eigenvalues of S was mini- 
mized in [63] and [3 ] . 

J = [ £ ]^^^ = [Tr (1) 

i=l i 

The worst case of the above, when k — > «>, is the 
maximum eigenvalue of S. The sequence of ^ - 1»2,.., 

are minimized in [ 3 ]. For k = 1, J = Tr [s] , and the above 
problem reduces to the same as the one presented in [ 39 ] • 
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Allwright considered a traceless formulation of per- 
formance criterion in L 2 ] and arrived at a Riccati-like 
differential equation, the solution of which leads to a 
closed loop system which will never be worse than the open 
loop system for any initial state, x^. In [ 1 ] Allwright 
found a sequence of feedback gain matrices Fj^ such that 
the performance of any is better than the previous one, 
Fj^_j^, for any initial state x^. 

Bengtsson et al. [ 9 ] adopted a method for design of 
systems with inaccessible states in which a dominant eigen- 
space of the closed loop system with state feedback design 
is tried to be retained as the feedback structure is reduced 
from state feedback structure to required constrained output 
feedback structure. More recently Medanic et al. [45] used 
projection techniques to retain a dominant invariant sub- 
space of the corresponding standard regulator in the output 
feedback solution. This method gives more insight into the 
properties of the corresponding closed loop regulators. 

[ 44 ] extends the method to discrete case. 

A transfer function approach in which the optimal 
systems transfer function is approximated by output feedback 
transfer function was presented by Therapos [57]. Eitelberg 
et al. [ 20 ] showed that optimal state feedback gains of a 
suitably reduced order model is almost optimal, as output 
feedback gains to the original system. 
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Much of the work in the area of output feedback regula- 
tors are based on the modelling of initial states as a 
random vector along the lines of the seminal paper by Levine 
and Athans [39] . 

Different methods were tried to solve the output feed- 
back regulator problem presented in [39] by several authors. 
Many took to general purpose gradient methods for the para- 
meter optimization. Davidon-Fletcher-Powell method (DFP 
method) was used in [16] and [28]; steepest descent method 
was used in [27J; conjugate gradient method was employed in 
[62]; and Broyden-Fletcher-Golub-Shanno method (BFGS method) 
was tried in [35], Newton's method based on second order 
Taylor series expansion of the loss function was applied to 
the problem in [60], with conjugate gradient method. 

The basic Levine-Athans algorithm, i.e. , solution of 
nonlinear equations, and its dual were presented with their 
global convergence properties in [58]. Anderson and Moore 
[ 4 J used linear set of equations from the necessary condi- 
tions to find an optimal output feedback. The original 
Anderson-Moore algorithm was, however, found to diverge, as 
shown in [56]. 

Modified versions of the algorithm were presented by 
Moerder et al. [43] and Makila [41] and convergence of the 
methods with mild assumptions were proved. Several methods 
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for finding optimal constant output feedback gains and their 
connections to loss function expansion are discussed in [6l]. 

O'Reilly [ 50] considers discrete time deterministic 
systems. Discrete-time stochastic systems are considered in 
[ 22 ], [41 J and [61] and continuous time stochastic systems 
in [37]. 

It was shown by Kuhn et al. [35] that a deterministic 
problem with initial states and disturbances at the output mode- 
lled as random vectors and stochastic problems lead to the 
same optimal output feedback law. 

One major disadvantage of all the algorithms is that 
they require a stabilizing feedback as the initial guess. 

This problem was circumvented by formulating an unconstrained 
optimization problem formed by augmenting the quadratic per- 
formance index with the structure constraints as an equality 
constraint by Choi et al. [l7], Wenk and Knapp [62] and 
Shapiro et al. [65], Bingulac et al. [ 10] solved a sequence 
of auxiliary optimal problems starting with full state feed- 
back to get the solution. The above methods assumes that a 
stabilizing state feedback is available. With the available 
methods, findiag a stabilizing state feedback is not a 
difficult problem. 

In many practical situations, the feedback elements may 
be constrained to be zeroes or linearly related to other 
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elements. Such constrained structure was applied to flight 
control systems in [55]; problems in robotics in [62]; to 
large systems, in which costs of inter subsystem communica- 
tions are significant, in [40]; in which subsystems are 
weakly coupled in [62] and to decentralized system in [59]. 
Necessary conditions for solution of linear quadratic pro- 
blems with multiple structure constraints on state feedback, 
which extend directly to output feedback cases, were develop- 
ed in [34]. Arkun et al. [ 5 ] gave lower and upper bounds of 
structure constrained controller costs which can be used for 
comparative assessment of different structures. 

Kurtaran et al. [36] and El-Sayed [21 ] gave methods to 
choose an optimal pair of observation matrix and feedback 
matrix for a fixed number of outputs, for the cases when 
observation matrix, C, is free to be chosen. A lower bound 
of cost for any 1-output system is thus found in [36]. Opti- 
mization of sensor locations subject to constraints is done 
in [ 23 . 

Much of the studies in output feedback problem has been 
done with infinite time problems and very little with finite 
time problems. Finite time regulators for continuous time 
stochastic system are studied in [37] and for discrete time 
stochastic case in [22], with the feedback gains varying with 
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In [13], regulation is done in finite time using a class 
of controls constrained to be the set of linear functions of 
outputs sampled at discrete instants of time and the feedback 
gains computed on-line. 

For static feedback gains solution to infinite time 
problem may not exist. Mullis [49] showed that periodically 
varying gains always exist so as to stabilize the system. 

When static feedback gains fail to find a solution to 
the problem, compensators are added till a satisfactory per- 
formance is attained. The results of static feedback case 
were extended to dynamic output feedback case by Johnson 
et al. [31] by augmenting the system states with compensa- 
tor states. The procedure was extended to different types 
of systems by several authors [47]. Efficient algorithms 
to find an optimal dynamic output feedback compensator of 
fixed order based on the above procedure are given in [35] 
and [62]. 

To translate performance requirements like overshoot, 
settling time etc. time multiplied performance indices were 
used in [64] for continuous time case and in [ 14] and [38] 
for discrete time case. 

In [15] theory of optimal output feedback control is 
extended to a class of quadratic performance measures whose 
weighting matrices are functions of frequency. 
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Since the existence of a solution to the optimal output 
feedback problem depends upon existence of a stabilizing 
feedback, results in output feedback stabilization like the 
one in [I2j are of much interest. 

Current research in the area of optimal output feedback 
are robust output feedback control and optimal controls for 
two-time scale systems [32]. 



CHAPTER 3 


STATIC OUTPUT FEEDBACK 

3.1 INTRODUCTION 

As discussed in the previous chapter, the static output 
feedback problem has been studied in great detail resulting 
in many methods for the solution to the problem considering 
cost, availability of computing system etc. With the 
different methods having their own advantages and disadvanta- 
ges. The epithet 'existing most efficient algorithm* does 
not fit any method. However, some of the methods have found 
wider application owing to factors like simplicity, amenability 
to use of modern digital computers, etc. 

The algorithm in this thesis is similar to modified 
descent Anderson Moore algorithm (also called descent mapping 
algorithm) used in [41] [48]. The general output feedback 
problem, with system having direct feedthrough, quadratic cost 
having cross weighting of state and control and feedback gain 
matrix to be of constrained structure is considered here. 

3.2 OUTPUT FEEDBACK REGULATOR PROBLEM 
3.2.1 Continuous time systems 

Consider the linear time-invariant continuous time system. 
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X (t) =* A x(t) 

+ B u(t) 

(1) 

X (t) = C x(t) 

+ D u(t) 

(2) 

Y.p(t) = CpX(t) 

+ DpU(t) 

(3) 


where 

£€ is the system vector, u € r''*^ the control inputs, 

1 

X€R , measured outputs and R ^ outputs to be controlled. 
A, B, C, Cp, D and Dp are real constant matrices of compatible 
dimensions. 

Realization (A, B, C) is assumed to be complete (com- 
plete controllability and complete observability) with 
rank [cj = 1 and (A, B, Cp) is assumed to be complete with 
rank [Cp] = Ip. 

The system, (1) - (3), is to be controlled by means of 
constant output feedback. Thus, u(t) is to be realised as 

u(t) = -F x(t) (4) 

where F£ such that the index of performance or cost 

function 

09 

J(F) = / (y^ Qj^yp + Su'Tj^yp + u'Ru)dt + y(F) (5) 
is minimised. 

y(F) is a scalar penalty function on feedback matrix 
elements having a continuous gradient with respect to F 
and for which J is bounded below for all Fs which render 
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closed loop dynamics asymptotically stable, and Q^, Rj^ and 
Tj^ are matrices satisfying the conditions, = Q]^ >, 0, 

Rj^ = Ri > 0, and - T![ > 0. 

The minimization of the objective function is performed 
in the set Sp of stabilizing feedback gains 

Sp = I f(A-BFC)€C“) (6) 

where 

Fi • ) denotes the spectrum, and c” denotes open left- 
half complex plane. 

A necessary condition for existence of solution to the 

output feedback problem defined above is that the minimizing 

* » 

F should make the system asymptotically stable. If FJtSp, 

the index 'J’ may tend to infinity. Stabilizability and 
detectability conditions assumed for standard regulator 
problem do not, in this case, guarantee stability [51 J. The 
existence of a stabilizing feedback 'F' is a non-trivial 
problem, a satisfactory solution to which is not yet availa- 
ble. 

Sufficient conditions for existance of solution to 
output feedback regulator problem are given in [41j, in 
terms of compactness of the level set as defined below 


=« (F^Sp 1 J(F) < p) 


(7) 
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3.2,2 Discrete Time Systems 

Consider the linear discrete time— invariant system 
described by 

*k+l = A (8) 

yfc “ ^ ^ 

yp “ ^p^k '^p^k ^ 

k 

where 

is the state vector, is control vector, 

1 

y^^R is measurement vector and y- ^ E ^ is regulated 

Pjj; 

output vector. It is assumed that realizations (A., B, C) 
and (A, B, C ) are complete and that C and C are of full 

r' n 

rank. 

Input Uj^ is restricted to be a linear constant feedback 
of the instantaneously available measurements Therefore, 

the set of admissible controls is defined by 


“k ^k • ••• 


(11) 


The index of performance of the system is 


J = ^ Qi y.. + 2u* Ti y„ + u’ 


:=o ^Pk "Pk ^ 'P 


{ 12 ) 


wh ere 


Ql — >1 O and R “R* > 0 are 


symmetric matrices, and ^ 
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Minimization of the performance index (12) is possible 
if and only if the set of stabilizing feedback gains, Sp 
shown in (13), is non empty 

Spt = I /(A-BFC) < 1) (13) 

where 

f { ' ) denotes spectral radius. 

3.3 NECESSARY CONDITIONS FOR SOLUTION TO THE PROBLEM 

Performance index (5) is transformed into state regu- 
lator problem using (3), 

OO 

J = J (jc' Q 21 + u.' Tj^ + R^)dt + y(F) (14) 

o 

where 

Q = S S » ^ S “^1 s 

R = Ql Dp + Tj^ Dp + D^ T]^ + Rj^ 

Using (2) and (4), the state equation (1) becomes 


x(t) = A x(t) - B(Ijjj + FD)"^ F C x(t) 


Defining = (I^j^ + FD)"^ F 

(15) 

x(t) = (A - BFj^C) x(t) 

(16) 

and u(t) - -F^C x(t) 

(17) 


Thus solution to the problem for a system having 
direct feedthrough, exists only if (1^^^ + FD)”^ exists. 

Substituting for u in (14) from (17), and using the 
transition matrix of the system (i)(t), (14) becomes 
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J = x; / ((t)'(t) (Q+C'F'T) + T'F.C+C'F^Riy) (J)(t)) dt x + y(F, ) 
o XX Ox 

(18) 

where 

Xq = x(o) is the initial state vector of the system. 

^ (A-BF,C)t 

For system (16), (t)(t) = e . 

Defining 

“ (A-BF,CV t (A-BF,C)(t) 

S = J ie ^ (Q+C'FJ^T+T'Fj^C+C'FiRFj^C) e ^ )dt 

(19) 

(18) becomes 

J = xi S Xq + y(F;l) = i^o + ^(^1^ 
where 

Tr(.) denotes trace of a square matrix. 

S satisfies the Lyapunov equation [29] 

A^’S + S A^ + W = 0 
c c 

where 

A^ = A-BFC and W » QK)' F^T+TFj^C+C F^RFj^C. 

Also, from (14), using trace operation 

OO 

J= / Tr ([CH-C'F1T+T’F,C+C'F*RF,C] X xV dt 
o !''■ 

Interchanging integration and trace operations and 

OO ' ■ 

substituting P = / (x xV dt , (23) 

’ o 

J = Tr([C^C'Fp+T'Fj^C+C'F’^RFj_C] P) = TrCwP] (24) 

where, W is as in 


( 20 ) 


( 21 ) 


( 22 ) 
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P also satisfies the Lyapunov matrix equation 

A^P + PA^' + X x» = 0 (25) 

C C “"O "“0 

where 

= A - BFj^C . 

Proofs for the above and the necessary conditions for 
the solution to be stated later are given by different 
authors [29], [33], [46], [63]. 

So without getting embroiled into the deta.ils of proof 
we can state the necessary conditions, which is given as a 
theorem. 

Theorem 3.3.1 [39] 

For the systems (1) - (3), optimal control in the 
context of optimization of J (5) is given by 

Uopt(t> = -P* v(t) 

with 

F* = F| (Ij^ - DF*)"^ (26) 

where 

Fj^ satisfies the condition rank [l - DF^^] = 1, and 
F* = ((B’S + T) PC' - 1/2 y'/Fj^)) (CPC)”^ (27) 

where 

dY(Fi) 

Y* (F^) = — and S = S' and P = P' are the solutions 

of the Lyapunov equations 
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A^' S + SA^ + W = 0 (27a) 

A^P + PA' + x* = 0 (27b) 

C ““O "“Q 

with 

A^ = A-BFj^C and W = Q + C'F^T + T' F^^C + C F]^ RFj^C. 

Since R > 0, W > 0 and S being the solution of Lyapunov 
equation, for A strictly Hurwitz S > 0. Similarly, for 
Xo Xq > 0» P > 0. 

y(Fj^) is used instead of y(F) because of computational 
ease. For D = 0, Fj^ = F. 

Remark 3.3.1 

In (3) if Cp = I and Dp = 0, the problem becomes a 
state regulator problem. 

oo 

J = / (x'C^ + 2 u'Tx + ul Ru) dt + y(F^) (28) 

o 

Remark 3.3.2 

In normal cases, the cross-weighting terms are not 
considered, i.e., Tj|^ = 0. 

Then, from (28), 

■OO ^ 

J = / (x’ + u’Ru) dt + y(Fi) (29) 

o 

and for y(Fil) = 0, (27) become 
= r“^ B’ SPC (CPC )~^ 


( 30 ) 
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Remark 3,3.3 

For systems without direct feedthrough D = 0 in (2) 
and =* F. Thus the problem reduces to finding the 
feedback gain matrix F such that 

u = -Fy = -F Cx . (31) 

Remark 3.3.4 

In (27) (CPC* )''^ exists because P > 0 and rank [c] =* 1. 

If 1 = n, result is that of optimal regulator with 
state feedback, with 

F = r”^ (B'S + T) C”^ (32) 

A*S + SA + Q - (B'S + T)’ R“^(B'S + T) =0 . 

For T = 0, and C = I 

F = -R*"^ B'S (33) 

A'S + SA + Q - S'B'R”^BS= 0 (34) 

which are the well known results of standard LQR theory. 

Thus if C exists, dependency of F on P, cf. (27) 
and (32), and thus on initial state x^, cf (27b), disappears. 

Remark 3.3.5 

Equations (27) and (27b) show that feedback matrix F 
is dependent on the initial state x^. Since initial state 
Xq may not be known a priori, this dependence is undesirable. 
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As mentioned earlier in Chapters 1 and 2, this problem 

can be dealt with by considering a random vector with 

covariance matrix X . The problem is then to minimise 

cov 

expected value of cost, J, i.e., to find a control to 
minimize 

oo 

J = E [ / (yip yp + 2 u* Tj, yp + u' R jlU) dtj (35) 

= E [x^ S XqJ = E [Ir is Xq x^)3 

= Tr(SXQ) (36) 


where 

X^ = E(x^ X* ) is second order moment of x_. 
O o — O 

(27b) becomes 


P A^’ + A^P + X« = 0 
c c o 


Xq =E(Xo Xq') =E((Xo-Xq) (Xq-^o^’ )+^o ^o 


+ x_ x* 
cov o o 


where 


x« = e[x ] 


For x^ = 0, X_ = X__„ 
o ^ o cov 


(37) 

(38) 


Kuhn et al. suggest X^ to be chosen as 

Xo = “*^*9 (1/4 I’^ioUax ”4=<i“4l 

values for independent identically distributed, initial 
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states with Gaussian distribution. 

Remark 3.3.6 

For a stochastic system, with plant disturbances ' , 

and system described by 

X = Ajc + Bu + G X) (39) 

X = C2c 

same set of equations as in Theorem 3.3.1 can be used for 
solution with substituted by GVG' , where V is the 
covariance matrix for the white noise disturbances ’ ’ . 

In this case, the cost (36) become 

J = Tr [S GVG* J (40) 

3.3.2 Discrete time Case 

Along the lines of continuous-time case necessary 
conditions for the solution of the discrete-time case can 
be obtained as a set of coupled nonlinear matrix equations. 

Thus the performance index (12) for T = 0 can be 
written as 

J = Tr [S Xq x;3 (41) 

where 

S» I [(A-BF,C)‘']' (Q+C'F^RFj^C) [(A-BFj^C)^] 
k-o 
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is the symmetric positive definite solution of the discrete 
Lyapunov matrix equation, 

S = (A-BFj^C)’S (A-BF^C) + Q + C'F^RFj^C (42) 

Using the expression for S in (41) J can also be written as 
J = Tr [(Q+C'FiRFj^C) Pj (43) 

where 

» <» . . 

P = ^ ik Xv = (A-BF,C)*^ X xL [(A-BF,C)^]’ 

k=o k=o ^ ° ^ 

is the symmetric positive definite solution of the discrete 

Lyapunov equation. 

P = (A-BFj^C) P(A-BFj^C)’ + Xq 

The corresponding optimal feedback gain matrix F^ is given 
by 

= (R+B'SB)”^ B'SAPC’ (CPC )”^ (45) 

With cross weighting term T and penalty on Fj^, Y(Pi)» 

= (R+B»SB)"^ [(B’SA+T) PC - l/2 y'(Fi)3 (CPC (46) 

The equations (45), (44) and (42) constitute the 
necessary conditions for the existence of optimal output 
feedback. 

All the remarks in the previous section can be con- 
sider4d for discrete-time case too. 

The necessary condition presented in Sections 3.3.1 
and 3.3.2 are a set of nonlinear coupled equations, solution 
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of which yields the optimal output feedback. The condi- 
tions being only necessary ones and not sufficient, there 
will be solutions of these equations which are not optimal. 

3.4 STRUCTURE CONSTRAINED REGULATORS 

Sometimes it becomes necessary to select a particular 
structure for the feedback gain matrix to make each control 
variable depend on a different set of output variables. 
Constrained feedback structure leads to considerable 
decrease in complexity for minimal deterioration in perfor- 
mance [55] . This is of special significance when some 
elements of the feedback gain matrix are nofc-productive [40]. 
For example, in large systems, non-zero elements of feed- 
back matrix entails all measurements be available to all 
subsystems, but the cost of the required communication links 
may outweigh the benefits gained from them. 

Several structures can perform close to the uncon- 
strained regulators. 

As in the case of unconstrained output feedback, the 
system may not be stabilizable with some or any of the 
constrained feedback matrices; only a few of them may be 
realizable. 

If Sc denotes the set of stabilizing feedback gain 
/^c 

matrices with allowable structures, 
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Sp (FeTc 1 /(A-BFC)€c“) (47) 

c 

where 

Tfcis the set of feedback gains with allowable feed- 
back structure. 

Sp d Sp • 

3.5 MODIFIED DESCENT ANDERSON MOORE ALGORITHM 

The set of equations representing the necessary condi- 
tions in Theorem 3.3.1, can be solved in several ways. 

Levine and Athans [39] who derived these necessary conditions 
and several others [56], [58] solved the nonlinear set of 
matrix equations to arrive at a solution. But solving non- 
linear equations is difficult and computationally demanding, 
although the rate of convergence is good as shown in [58], 
for some examples. 

Anderson and Moore [ 4 ] solved coupled linear equa- 
tions iteratively to get the solution. With an initial 
feedback gain matrix, F^^ Sp sequence ^Fj^j is generated 
as, 

Fj^^l = -R"^B'Sj^Pj^C’(CPj^C' )"^ (48) 

where 

^k “ ^^^k^ ^k ~ the solutions of 

Lyapunov equations (27a) and (27b) respectively. 
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The above algorithm, however, was found to diverge [56j. 

An alternate way is to find a sequence of gains 
generated by 


^k+1 ” “k ^k 


(49) 


where 

> 0 is a step length parameter, Fj^£ Sp and 
is a descent direction. 


In the algorithm employed here, is selected so as 
to minimise a quadratic approximation to the loss increment 
as given in equation (76) [4lJ. 

I 

The search direction is a significant factor in deciding i 
the speed of convergence of the algorithm. | 

decides the step size in each iteration and has to | 

^ I 

be selected judiciously to improve the efficiency of the 

I 

algorithm in terms of the number of loss function evalua- | 
tions, iterations, etc. In this algorithm is selected | 
in each iteration using quadratic fitting, so as to satisfy 
the Goldstein steplength rule [24]. 

When the feedback structure is constrained each con- 
trol input is generated as 

u ( t ) =— F £ (t), i = l,2, •*., m ( 50 ) j 

where 

^i “ ^ = 1,2, m 

is a submatrix of C corresponding to vector 
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Kosut [ 34 ] calls this as minimal structure constraints 
problem. 

Each row of the feedback matrix ’ F' is given by 
Fi = ~ b!_ SPq (qPG[)“^ (51) 

where 

r^ » R(i,i), R is assumed diagonal, and b^ is the 
ith column of B. 

Based on the above discussion a stepwise procedure is 
given below to iteratively compute a static output feedback 
gain matrix with and without structure constraints. 

Algorithm 
Step 1 

Choose Cg€(0, 1/2), > O, > 1, 

fmin ^ Jmax ^ ^2 > 0- "here and 

^2 arbitrarily small values to be decided by the desig- 
ner depending upon the accuracy of solution required. 

Set k = 0. 

Step 2 

Find an initial stabilizing feedback F^ with structure 
constraints. 

Step 3^ ^ ^ ^ ^ ^ ^ 

Solve for S and P using (27a) and (27b). 
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Step 4 

Compute 

“ 2[rFCPC' - (B’S-HT) PC'] + y'(F) (52) 

= R"^[(B'S+T)PC' - 1/2 Y*(F)j (CPC )”^ -Fj^ (53) 

Fox multiple structure constraints, 

AF^ = ~ (b[S + T) PC^ (Cj^PCp-^ - F^ (54) 

where 

AF^ is the ith row of AF. 

Step 5 

Test for termination. 

If 1 |AF| 1 < ^ 2^ , go to step 9 : Otherwise continue. 

Step 6 

Determine by finite research process. 

al) Set a = Determine smallest integer j, 

.0 £ ji < jmax* 

J(F^) - J(Fk + “j < -d-Cg) "‘j Tr [M; H^] 
where 

*j+l = ■"1" [^Bin “j • 'I'J ’ ^max > 

where 

+w if ^ _< 0 

af 

- Tr[(|^)|^ Hj^] if f > 0 



(55) 

(56) 

(57) 


4 ) 
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wh ere 

^ Hj^) - J(F^) - a. Tr [(1^)^ Hj^] (58) 

a2) If j = 0 go to step 6b). 

If j ^ 0 go to 6c. 

6b) Set ® determine the smallest j such that 

(Fj^ + ttj Hj^) ^ Sp and 

J(Fj^) - > -Cg ttj Tr L(^)it (59) 

where 

“j+l = [^in “j> I'J’ ^max 

$ being given by (57). 

6c) Select min J(Fj^+ a. H. ) 

“j 

Set a. » a . 

^min 

Step 7 

Test for termination. 

J(F^) - 

If j^jp- -^ — ^ 2 * go fo step 9 ; otherwise continue. 

Ic 

Step 8 

^k+1 “k k = k + 1. 

Go to Step 3. 

Step 9 

Set F* <— F(1-DF)“^ 

Remark 3.5.1 

J[ Y . \ i S a RPtI a « whi r-h #*ar\ t..4 4.u j.«- ^ 
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relation [ 42 ], 

^k+1 = Yj, + (1-u)) a^, Y) 

where 

0<a)<l, Yq = 1,0<Y<1 

Or set equel to 1, s 1, k = 0,1, ... 

Remark 3.5.2 


( 61 ) 


In the breakdown of CPU usage, function 

icrion and gradient 

evaluation dominate the solution time [231 ann 

L j and these are 

in turn subject to efficiency of solution of Lyap 
matrix equations. 

In the algorithm for solving (27a) and i+^ 

> ana its dual (27b), 

Bartel and Stewart algorithm [7 ] is the most • 

^ i c 1 *t 

method available. The algorithm is a direct 

rect procedure to 


solve equations of the form A'X + XA = -c 


is appli- 


cable to unstable As too. Belanger [8 J had shown that 

this algorithm was superior to other existinn 

Asxing ones with 

respect to accuracy and speed. A more efficient alg 'th 

for solving equation of the form A*X + XB = -r • 

^ Is developed 

by Golub et al. [25]. However, for solving Lypunov 
tions, there is no advantage over Bartel and Stewa t 
algorithm. 

In the algorithm, A is transformed into 

Schur form, 

which facilitates calculation of eigenvalues of ^h 
system for testing stability, without additional 
tational effort. 
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Remark 3.5.3 


Using equf^tions (40^ ^ 

>> (44) and (45) in place of (274), 

(271^ , and (27) the same alao-ri-i-K i. 

axgorithm can be extended to the 

discrete time case. 

3.6 INITIAL SIABILIZING FEEDBACK 

All algorithms requi-ro = . . 

Huire an initial stable feedback 

gain matrix to initiate th^ 

T^ne algorithm. The problem of 

finding such a priming feedba^u • ^ u 

9ck IS not, however, trivial. 

A stabilizing output . 

eedback gain matrix may be found 

using pole placement techn^«., r ■> 

[51] or cost techniques [ 6I] 

involving minimization of a -f . 

function of eigenvalues or 

coefficients of the charar+«„- . . 

^acteristic polynomial. 

The method adopted here ^ , . 

is different and is illustrated 

in a flow diagram in Fig. 

3.6.1 Minimum Norm and Minim 

minimum Error Excitation 

Minimum norm and minimum 

-Lnimum error excitation are found 

using the optimal state feeriha^i ^ . 

eedback matrix and the solution of 

associated alsebraic Rlccati equation [34]. since the 

guarantee that the resulting leedbaclc will be stabilizing 

is not there, this is onlv a + • n 

f trial to avoid more computa- 


tions. 




»3 


. ysed for finding 

Fig.l Flow diagram of the me1h° ^^^dback 
on initial sfabillzing outp^V 
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For the minimum norm feedback [34] the optimal state 
feedback solution K* found using (33) and (34) is put in the 
required output feedback structure form, retaining the uncon- 
strained elements. 

Minimum error excitation feedback [34], 

= K* PC| (C^PCp“^, i = 1,2, m (62) 

where is as in (50), is ith row of K the optimal state 
feedback and P is the solution of Lyapunov equation, 

(A-BK*)P + P(A-BK*)' + Xq = 0 (63) 

with x' if x^ is known a priori and 

E[Xq otherwise. 

K* is given by, 

K* = B'S 

where 

S is the solution of the algebraic Riccati equation, 

A'S + SA + Q - S'B'R"^BS =0 (64) 

Since (62) - (64) contain dynamic information only about the 
optimal system and not about suboptimal system, stability is 
not ensured. When above methods fail, constrained optimiza- 
tion discussed in Section 3.6*2 is done to find F^^ Sp. 

3.6.2 Gonstrained Optimization 

In this method, an additional constraint is added to 

the orioinal OroblAm and i 



using method of multipliers [26], 


In [l7], [55] and [62] the same method is used to 
optimize the constrained cost and find a solution to the 
problem. In this thesis only partial optimization is 
done till a stabilizing feedback is obtained. 

For required feedback structure F^, which has free 
elements and fixed elements, the error 0(F) is defined 
as 


r 

0((i» j) “ S 


F(i» j) constrained element in F 

0 for free elements in F 


( 65) 

where 


Fj(i,j) correspond to fixed elements in F^ 


Thus cost J cf. (36) is to be minimised subject to 
equality constraint (65). 


This classical equality constrained static optimiza- 
tion problem encountered here is solved with Hestenes 
method of multipliers [26]. This method is shown to be 
computationally simple and more stable than penalty 
function method [30]. 

Define the augmented Lagrangian, 
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where A is the multiplier matrix, y > 0 is a scalar constant 
and is as defined in (65)* 

For A = 0, J represents a straight penalty method. In 
this method, y has to be selected large to force 0 ===> 0. 
Computational difficulties due to rounding off error and 
resulting ill-conditioning of associated Hessian matrix arise 
when 0 gets small [62], Addition of multiplier term alleviates 
this problem. 

The algorithm for solving the unconstrained optimization 
problem in order to obtain Sp is presented below. In the 
algorithm, Broyden-Fletcher-Goldfarb-Shanno (BFGS method) is 
used since it is the most efficient method to solve unconstrain- 
ed optimization problems [l9 ] , [35]. It has the advantage of 
fast convergence rate but disadvantage of requirement of large 
storage space. 

Algorithm 
Step 1 

Matrix C is augmented by adding rows to make it a non- 
singular square matrix. 

Matrix F is augmented with additional columns to make it 
confirmable with C. 

With any stabilizing full state feedback, K 

. o 

F - is a stabilizing matrix 


(67) 
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Step 2 

Select > 1 

and set y > 2^^ j/Tr [(Zi’ (Zij (68) 

determines the emphasis placed early in the descent on 
satisfying the structural constraints on F. ^ that 

penalty term should be at least comparable to J. 

Select the number of descents in one cycle 

^2 > 1 and set A = 0, and i = 1. 

Step 3 

Set = I 
o 

where, I is an identity matrix of order 1^ x 1^. 1^ = mxn 

and H is the Hessian matrix. 

Step 4 

Solve for S and P with (27a), (27b) 

Find ^ + A+ Y 0(F) (69) 

where 

is given in (52), 0 in (65). 

Step 5 

If i = 0, go to Step 6 ; 


Otherwise update the Hessian matrix H. 


u Hj qjPi Pjq* Hj ^ r, ^ pi«i5i 1 Mi 

^ i+l ” ^i " p Pi^i 'Pl^i 


( 70 ) 
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where 

= col(F^^j^) - col(F^) 

Pi “ 

where 

col(.) is defined as : 

z = col(dj):= z' = [dj^ dj^, ..., dj^^] (71) 

where s are columns of dj. 

Step 6 

Find 

coKAF^) = col(|^) (72) 

where A F^ is the search direction matrix. 

Step 7 

Find step size parameter A method for finding such 

an is presented below. 

Stage 1 

a) Set = 0, = J(Fj^), k = 1 

b) Set = a^, = J(F^) 

If Fj^^ Sp, reduce to make Fj^ ^ Sp. 

where, Sp is the set of stabilizing feedback gains. 

Set k = k + 1. 

c) If Jj^ > 9® "to next step; otherwise reset = 2a^ 

and go to (b) 



d) If k > 2 go to next step; otherwise reset 
= a^/2 and k = 1 and go to (b). 


e) 


Set A - ® “ “k-1’ 

A+B+C B+B+C 
®o = “3 


Set 


a 


o* 


c = “k 

4 4^ A+B+C 
'~3 



and find 

= B. 


Stage 2 


f) Let d" g ” inin(J^j 

find 2 adjacent points to B and label them A and C. 
Ja - “^B - -^C* 

g) Compute a = — and 

O o o 

Set oc , = a_ and Jj = 

do do 

If lJs"'^d' ^ '^d » 

otherwise set = a^, and go to step (f). 

h) Find as , 

Ja* = Jd' 

Step 8 

Set 

A F^ and i = i+1 

Step 9 

If i k n^ where k = 1,2, ... go to step 4; 


otherwise continue 
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Step 10 

If F € Sp , where Sp is a set of stabilizing output 
feedback gains with the required structure. Set = F; 
otherwise update y snd /\ using (73) and (74) and go to Step 3. 


^j+1 ■ ^2^j 

(73) 

^j+1 0 (F^) 

(74) 


Remark 3.6.2. 1 

Step 10 is included to find a stabilizing F before 
optimization of cost function J is complete. If desired 
optimization can be done for more number of cycles, thus 
partially optimizing J, the original cost, cf. (36). 

Remark 3. 6. 2. 2 

The search method in Step 7, called arithmetic mean 
method, is found superior to RMS, quadratic and cubic inter- 
polations and golden section methods in terms of number of 
function evaluations and CPU time [54j. 

If this algorithm fails to find a stabilizing feedback 
the whole method fails. 

3.7 SEARCH DIRECTION 

The search direction cf. (31), decides the rate of 
convergence of the algorithm. If proper search direction is 
not found, algorithm may not even converge. 
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When search direction satisfies the descent condition 
Tr (|^)^ < 0 (75) 

converges in the descent direction 

[48 ] . 

Different methods have different search directions. In 
all the gradient methods, the search direction is derived 
from the gradient of the objective function. 


the sequence 




The search direction used in the algorithm presented here 
is also a gradient based one and it is found to minimize a 
wquadratic approximation of the loss increment shown below. 

The search direction is also found to satisfy the descent 
direction condition [48]. 

The loss increment J(F+AF) can be expressed as [41], 

[60] 

J(F+AF) = J(F)+2 Tr [A F' [-B'S(F+AF) + RFC] PC*] 

+ Tr [AF* RAFC P(F) C* ] (76) 

A Taylor series expansion for loss gives, 

J(F+AF) = J(F) + q(AF) + o(| lAFll)^ (77) 

where 

q(AF) = Tr IAF’ (|^)] + Tr[AF* rAFC P(F) C* ^ ^ 

(78) 


+ 2 Tr [A F* B’ AS( AF) P(F) C’ ] 
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S(AF) is the first order Taylor series form in F of S(F). 

A positive definite approximation to q (AF) obtained by 
ignoring the last term in (78), i.e., taking S(Af) = 0 or 

S(F+Af) = S(F) (79) 

gives 

q(AF) = Tr [AF* (|^)3 + Tr[AF' rAFC P(F) C j (80) 

The search direction is computed to minimize the incre- 
ment q(AF) with respect to F. 

A necessary condition for above is, 

dq(AF) 

= 0 (81) 

Thus, from (80) 

1^ + 2 rAF CP(F) C = 0 

i.e., Af = - 1/2 r“^ (^) (CPC’)"^ (82) 

with =* RFCPC - B’SPC 

Af = - 1/2 r"^ b'spc (cpc)"^ - f 

which is the same given in Theorem 3.3.1. 

If S(F+AF) in (76) is approximated by a Taylor series 
expansion, another set of equations involving S (AF) and 
P (Af) are obtained. Newton’s method of solving parameter 
linear quadratic control problems [60] uses these sets of 
equations for descent direction in each iteration. Although 
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Newton's method has quadratic convergence rate, it may be 
expensive and benefits being only local, may not be effective 
when far away from solution. 

Descent Anderson Moore method with finite search process 
finds a local minimum to J in (36) if one exists, as stated 
in the theorem given below. 

Theorem 3.7.1 [41J 

For nonempty Sp in (6) and compact level set (7), 
sequence generated by search direction A F in (82), and 

step length parameter in (49) which is chosen so as to 
satisfy Goldstein step-length rule with Sp, will con- 

verge to such that 

, 5 j X 

( — i.) ^ 0 for some k. 

Since Sp is an open set, any local minimizer of J(F) on Sp 
must be a stationary point of loss function J(F) on Sp. 

Descent Anderson Moore method has been widely used 
[15], [42], [48], [59] and [ 60] . It is shown in [59] that 
this method is superior in terms of number of iterations and 
number of loss function evaluations to some of the general 
purpose algorithms like steepest descent method, BFGS method 
with and without scaling of initial Hessian matrix H^. 
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Since the problem may have several local minima, for 
example, in the case when Sp is a set of disconnected regions, 
algorithm will have to be initiated from different initial 
values of feedback matrix to find the global minimum. 

3.7.1 Unidimensional Line Search 

Like research direction matrix, step size scaling is 
also an important factor in deciding the convergence and 
efficiency of the algorithm. An example which was found to 
diverge in [56] where step size scaling is not done, can be 
shown to converge with step size scaling. 

The algorithm presented in this thesis uses quadratic 
fitting with Goldstein step length rule and a modification 
of Armijo line search [ 6 ], for unidimensional search, cf . 

Step 6 in Section 3.5.1. 

Quadratic interpolation is done with three values known 
at the beginning of the search, 

(1) JCF^) (2) Tr [(|f)k AF^] and (3) J(F^+AF^) 

(3) above is the cost at Newton step, = 1. 

New value a is given by [19], [42] 

- Tr {[dJ/dF)!' A f^] 

a = — - — (83) 

2[J(Fj^4-AFj^) “ - Tr L(dj/dF\ AFj^]] 

The step size 'a* chosen should satisfy the Goldstein rule 
124], 
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-Cg Tr[(dj/dF);AF^) < J(F^) - J(F^+a^AF^) 

1 (i“Cg) Tr[(dj/dF)J^ AFj^J (84) 

The left hand side of the above ensures that average 
rate of descent is at least some prescribed fraction of the 
initial rate of descent in the search direction, thus avoid- 
ing very small decrease in cost values relative to the 
length of steps taken. 

Figure 2 shows the significance of judicious choice of 
Cg for greater descent. The best value of Cg, however, 
depends upon the particular problem. 

The sequence | Fj^j generated by using step sizes that 
satisfy the left hand side of (84), under certain assumptions, 
will converge to F , a local minimiser of cost J. These 
conditions were demonstrated separately by Armijo [6 J and 
Goldstein [24]. 

Unlike exact line searches, viz.. Golden section 
method, Newton’s method, curve fitting methods like cubic 
and quadratic interpolations, this method is an inexact line 
search without any minimization of function of single varia- 
ble. Although the exact line search gives a parameter for 
which cost is less than that for the parameter obtained by 
inexact line searches, the total number of function evalua- 
tions may be more. 
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3. 

8 CONVERGENCE CRITERION 



One of the following convergence 

criteria is employed 

in 

almost all algorithms. 


1) 

IIAfH < g 

(85) 

2) 

lldJ/dFll < 

(86) 

3) 


(87) 


1 /rs 



where l|x|l;= [Tr (X'X)]^'^^, ^ > 0 is a small value. 


Algorithms in [15], [58], use (85) and those in [l6j, 
[35], [42] and [59] use (86). 

For special cases, when J(Fj^) — >0» (87) should be 

iJ(Fj^^l) - J(Fj^)l < ^ (88) 

Such a criterion is used in [62]. 

It is suggested to use several criteria simultaneously 
for a general purpose algorithm [19], since different cri- 
teria will be suitable for different problems. Value of ^ 
is to be selected according to the accuracy requirement 
and effort demanded of the program in satisfying it. In the 
algorithm used here, (85) and (87) are used for testing con- 
vergence, cf. Section 3.5. 

3.9 CONCLUSIONS 

Several algorithms for the design of LQR with output 
feedback have been studied in this chapter. 



CHAPTER 4 


DYNAMIC OUTPUT FEEDBACK 

4.1 INTRODUCTION 

Traditional methods of servomechanism employed dynamic 
compensators like lag, lead or lag-lead networks to improve 
system performance. They were designed using root-locus 
and/or frequency domain methods. These graphical methods 
were suitable for single-input single-output systems but 
difficult to handle for multivariable systems. 

In the state-space approach, Kalman filter or Luenberger 
observer is used to estimate the states for feedback, depend- 
ing upon whether the system is stochastic or deterministic. 
But for high-order systems, order of these compensators are 
also high bringing complexity into the system. For the 
design of Kalman filter, covariance matrices for measurement 
and plant driving noises have to be specified which, however, 
is difficult for essentially deterministic systems. 

Compensators of order (n-1), where n is the number of 
states and 1 the number of outputs, are required for state 
estimation in reduced-order observers. For designing optimal 
feedback systems, a minimal order observer which estimates 
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a specified linear functional of system states can be used. 
Order of minimal order observers is less than that of reduced 
order observers. Observers require input to the system for 
its operation whereas there is no need for it for dynamic 
output feedback compensators. The dynamic output regulator 
approach avoids putting poles of compensators at infinity 
which is allowed in observers, thus not permitting high fre- 
quency plant driving noise to pass through. 

4.2 DYNAMIC OUTPUT FEEDBACK REGULATOR PROBLEM 

Consider a linear time-invariant continuous-time system 
described by the state and output equations, 


X = Ax + Bu 

(1) 

y = C X + Du 

(2) 

+ D„u 

-^P p— 

(3) 


where x ^ R*^ is the state vector, u ^ R® the input vector, 

” ”1 
y^R^ the measured output vector and R the regulated 

output vector of the system. 

Matrices A, B, C, D, C and D_ are fixed and of compa- 

IT H 

tible dimensions. . 

The system is to be regulated by a dynamic compensator 
of the form 

“ = Cc Xj + 1 


( 5 ) 
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n 

where ^ is the state vector of the compensator and 

Ac» and are constant matrices of compatible di- 

mension to be determined such that the index of performance 

oo 

j = E [ / Q Ip + T + u' R u + xj, x^ 

^ ^c2 ^p ^ ll> %3 ^c ^c) (6) 

where Q = Q’ > 0, R = R* >0, = ^cl ^ ° “ ^c ^ ° 

is minimised. 

A dynamic output feedback controller is illustrated in 
Figure 1. The order of the compensator n can be chosen 

w 

with the obvious restriction that a controller of the 
chosen order must be able to stabilize the plant. 

For solving the dynamic regulator problem described 
above, it is first transformed into the static constant 
output feedback problem as described below. 


4. -3 EQUIVALENT STATIC OUTPUT FEEDBACK PROBLEM 


The system state vector x is augmented with the com- 
pensator state vector x , Inputs to the augmented systems 
are u and x and outputs are x i£r* 


Thus defining x = 


21c 


» M = 


u 


ilc 


» 1 = 


^c 


yip = 


the system equations (1) - (3), and compensator equations 
(4) and (5) can be combined as 




/diOT. No. 
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X A -N A A 

X = A X + B u 


A A A A A 

1 = C X + D u 



A 

A 

A 

A A 






X + 

D u 

P - 



where 







A 


A 

0 ' 

A 

r* 

B 

0^ 

A 


_ 0 

0 

..J 

, B = 

0 

I 










~ C 

0 ^ 



o“ 

A 




A 



C 


_0 

I _ 

, D = 

_^0 

0_^ 



r- 





A 


s 

0 

^ 1 


0~ 



_0 

I 

II 

0 

0 


The control to be realised become^ 

A A 

u =-F Y. 

where 



and the index of performance (6) reduces to 

/V A A A A A A A 

J = E [ / Q + u' T U» R u) dt] 


( 7 ) 

( 8 ) 

( 9 ) 


( 10 ) 


( 11 ) 
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wh ere 


> 0, R = 


R O 


U R 


When Xq and x^o* initial states of the system and com- 
pensator are assumed to be random vectors with 

>(0 = E [Xo 4]. Xco3 • 

Second order statistics of initial states of augmented 
system, _ 


, where and x^^ are 


= 


assumed independent. 

It can be seen from system equations (7) - (9), con- 
trol equation (10) and performance index (11) that the 
problem now is equivalent to a static output feedback pro- 
blem with feedback gain matrix 


F = 


This problem can be solved using techniques described 
in Chapter 3. 


While designing dynamic regulators in practical situa- 
tions with realistic plants, certain difficulties arise p5], 
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One such difficulty is due to the redundancy in the number 

of parameters free to be designed. State-space description, 

with elements of A , B , C_ and D_, contains more free ele- 

c c c c 

ments than the minimum number required to present an arbi- 
trary input-output behaviour. When number of free para- 
meters are more, computational effort required is more. 
Constraining certain elements in the compensator matrices 
alleviates this problem to some extent. 

In certain cases an optimal regulator is resulted which 
is partly decoupled from the plant. If the algorithm is 
started with a lower order compensator, the optimal solution 
cannot be a higher order compensator, since a minimum of J 
exists for every compensator order [35], With the parti- 
cular structure of block matrices in (7) - (10) time required 
for computation is more. Use of penalty on feedback elements 
will particularly be useful in the design of dynamic com- 
pensators. 

4.4 CONCLUSIONS 

The problem of designing LQ regulator using dynamic 
output feedback is presented in this chapter. Computational 
difficulties that arise during the design of dynamic output 
feedback regulators are also discussed. 



CHAPTER 5 


GUIDELINES FOR USE OF THE DESIGN PACKAGE 

Programs using the algorithms presented in the previous 
chapters have been coded in Fortran code. It can be used 
for designing linear quadratic regulators with state feedback 
or output feedback, with or without contraints on the feed- 
back structure. The program is user-oriented, with fixed 
inputs to be provided for the problem specification and 
design parameter to be chosen by the user, Since the most 
suitable values of these parameters differ for different 
problems, the designer may have to try more than once to get 
a fast convergent solution. Since the solution may correspond 
to a local minimum, the algorithm should be initialized from 
different values of the initial stabilizing feedback gain 
matrix F^, to get a global minimum. 

Following are the inputs "tequired to specify the pro- 
blem. These are provided at the start of the program. 

n = order of the plant 
P 

mp = number of inputs to the plant 
* 1 = number of outputs of the plant used for controlling 

' ■ ii# ■ 

the plant 

1 = number of outputs to be regulated 

P 
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N 


sig 


A 

B 

C 

D 


D. 


R 

X, 


= number of significant digits in the elements of A 

which determines the convergence criterion €^2 

the reduction of closed loop matrix A to Schur form. 
-N , ^ 

6 2 1 10 

= system matrix of order n^xn^ 

P P 

= input matrix of order HpXnip 
= measurement output matrix of order l^^xnp 
= direct feed through to measured outputs of order 

Icxmp 

= regulated output matrix of order l^xn 

Jr r 

= direct feed through to regulated outputs of order 

1„ xm„ 

P P 

= weighting matrix on regulated outputs of order 

l_xl 
P P 

= cross weighting matrix on regulated outputs and 

controls, of order m„xl„ 

= weighting matrix on controls of order nip xmp 

= E(x xi) - second order statistics of the initial 
o o 


state vector 


The user can decide the order of compensator n^'. When 
n^ is given as zero, static optimal output feedback gains are 
found by the program. When n^ > 0, following matrices should 
be given by the user* 
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Qc 3 ~ cross weighting of system states and compensator 
states; of order ^pXn^ 

Qj. 2 ~ cross weighting of compensator states and system 

states, of order n^xn„ 

c p 

Qci = weighting matrix on the compensator states, 

of order n^xn„ 
c c 

Rc ~ weighting on x^, where Xj. ^re the compensator 
states 

Xco =* second order statistics of the compensator states. 

The user can decide the penalty on feedback matrix 
y(F). The penalty function employed in the program is of 
the form 


m l « 

y(F) = 2 Z L(i,j) [F(i,j)]^ 

i=l j=l 


where F is the feedback gain matrix and L, is a matrix of 

order mxl to be provided by the user, m = m +n and 

p c 

1 = l^+n_ are the number of inputs and number of outputs of 

C C 

the augmented system. Each element of L decides the penalty 
on the corresponding element of F. 


The design parameters to be provided are given below 
with recommended values and recoimnended range of values for 
each [42] . 



= This valye determines the step length in each 
iteration of the algorithm. 
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Although in Goldstein's criterion, C €(0, 1/2), 

y 

recommended values are C £ (0.05, 0.3). 

y 

^max = = 2(1.5, 3) such that Z^^^ > Z^.^ > 1. 


These values are used for quadratic fitting in Step 6al 
of algorithm in Sec. 3.5. 


Jjjjax “ 2(0,3) decides the number of times (55) in Chapter 3 
is checked. When = 0, the step 6al is skipped 

which, however, does not affect the convergence 
property of the algorithm. 

Pmin = 0. 2(0.1, 0.3) and = 0.35(0.2, 0.5) such that 

lllxil Ilia X 

•^min — '^max 

These values are used for quadratic fitting in Step 6b 
of algorithm in Section 3.5. 

w and Y = such that 0 < (»),Y < !• These values are used to 
update the sequence Yj^ • 

^ = A small parameter which decides the convergence 

criterion using llAFj^ll, cf. (53) in Section 3.5. 
Value of that should be given depends upon the accuracy 
wanted by the user, > 0 . 

^2 = A small parameter which decides the convergence 

criterion using normalised change in cost for two 
consecutive iterations, Since suitable 

values of this parameters vary widely for different 
problems, care should be taken in deciding the 

value. 


CHAPTER 6 


RESULTS AND DISCUSSION 


It is shown, in this chapter, how the method employed 
in this thesis fares with other approaches to the problem 
and other existing methods of solving the problem. The 
importance of choosing appropriate values for certain de- 
sign parameters is also discussed. The system description, 
the weighting matrices etc. for the examples discussed in 
this chapter are given in the Appendix. 

It was observed in Chapter 2 that Therapos [57] used 
approximation of state feedback closed-loop transfer fun- 
ction to find an output feedback regulator. For the system 
described in Example 1, he obtained a feedback matrix Fj 

shown in Table 1. Sj is the corresponding cost matrix. 

* 

Using the method in this thesis, an optimum feedback F was 
obtained with the corresponding cost matrix S . It can be 
observed that but for a few elements, S* has elements with 
values smaller than that of S^. Thus for almost all initial- 
state conditions, costs will be lesser when F* is used for 
feedback. 

But as shown by Allwright [ 1 ] the minimization of 
expected value of cost does not guarantee a closed-loop 
system which can perform better than the open-loop system 
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for all initial state conditions. The cost matrix for the 
open— loop system, S^, and that for the closed— loop system 
with optimal output feedback, S*, are shown in Table 2 for 
Example 2, For an initial state [l 0 O], open-loop cost 

is less than that of the optimal closed-loop system. The 
closed-loop system obtained by Allwright [3 ] after minimi- 
zing a performance index 

J = [Tr ([S(F)]Pj^/P for p = 11, 

suffers from the same drawback, as shown in Table 2 for 
Xq = [O 1 Oj. A method to find a dominant feedback, a 
feedback which results in a closed-loop system with an index 
of performance better, for all permissible initial state 
conditions, than that for a closed-loop system formed by 
any other stabilizing feedback, does not exist, at present. 

The descent Anderson, Moore (DAM) method with a good 
line search has shown good convergence properties. Table 3 
and Table 4 shows this algorithm being compared to the 
Levine-Athans method and its dual algorithm in L^S] for 
Examples 3 and 4. DAM method shows better convergence than 
both algorithms for Example 3 and than Levine-Athans algorithm 
for Example 4. Dual Levine-Athans method converges fast in 
the first few iterations after which it is the same as that 
for DAM method. Davidon-'Fletcher-Powell (DFP) algorithm 
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employed by Choi and Sirisena [16] to find the optimal out- 
put feedback solution for Example 4, took 35 iterations to 

reach the convergence level E (dj/dF.)^ < 0.1, with an 

i ^ 

index of performance 159.12. With the same initial feedback 
gain matrix and convergence level, DAM took only 10 itera- 
tions to converge to an optimal cost of 159.09. 

Table 5 compares DAM method with steepest descent method 
which uses gradient of cost with respect to the feedback gain 
matrix as the search direction, for Example 4. 

Using the modification given by Kosut [34l, for diagonal 
R (the weighting matrix on inputs), the algorithm is adapted 
to finding optimal structure constrained feedback. For 
Example 5, used by Moerder et al. [48] to illustrate use of 
penalty on feedback elements, it took only 12 iterations to 
converge to the feedback gain matrix given in [48]. The 
penalty method of forcing feedback elements to zero is 
similar to straight penalty method and may involve computa- 
tional difficulties. 

For unstable systems as well as stable open loop sys- 
tems, minimum norm and minimum error excitation feedbacks 
can be good priming feedback gain matrices. For Example 1, 
when it took 7 iterations with initial feedback gain matrix 
Fq =0, it took only 3 iterations when minimum norm feedback 
was used as F^. For example 7 with F^ same as the one used 
in [59] (given in Table 9), it took 5 iterations and 



62 


20 function evaluations whereas with minimum norm feedback 
F^, cf . Table 9, it took 4 iterations and 15 cost fun- 
ction evaluations. For the unstable system described by 
Example 8, minimum norm feedback was found to be effective as 
Fq* algorithm converging in 9 iterations. Same problem 
was solved by Choi and Sirisena [ 17] using unconstrained 
optimization. Since details regarding number of iterations, 
number of function evaluations etc. are not given in [17], 
comparison was not possible. 

The algorithm presented in this thesis requires an 
initial stabilizing feedback having the required structure, 
which can be obtained using methods presented in Section 3.6. 
Using minimum norm and minimum error excitation feedbacks, 
cf. Section 3.6.1, first level unconstrained optimization done 
in [62] can be avoided in some examples. 

DAM algorithm, like other algorithms, shows poor con- 
vergence properties for dynamic output feedback regulator 
problems. It was observed that with = O, where 

B and C are the initial compensator matrices B„ and 
CO CO c c 

in F^, the solution also had B_*- = C_* = 0, i.e., the com- 
pensator was decoupled from the plant. The dynamic compensa- 
tor obtained for Example 5 are shown in Table 10. Results 
show that when B^^ = 7^ 0, B^* = ,4 0. Since weighting 

is done on compensator states and its derivatives , optimum 
value of cost may be larger with higher order con^ensators 
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but the associated closed loop system may have better res- 
ponse characteristics. 

Several parameters are to be provided by the designer 
in the modified DAM algorithm, cf. Chapter 5. One of them is 
Cg which determines the Goldstein's step length criterion. 
This value affects the total number of cost function evalua- 
tions to a great extent. Tables 6 and 7 show number of 
iterations, number of times matrix Lyapunov equation (MLE) 
is solved and CPU time for different values of C , for exam- 
pies 3 and 5, respectively. Table 7 shows, in addition, cost 
J(Fj^) and | j at k = 5 and k = 10. Optimum value of Cg 

for Example 3 is near 0.2 whereas for Example 5, a value near 
0.03 gives better performance. It can also be noted from 
Tables 6 and 7 that CPU time is directly proportional to the 
number of matrix Lyapunov equations (MLE) solved. 

is a series which can be generated using the rela- 
tion (61) in Chapter 3 or set equal to 1 for all iterations. 
Table 8 compares the two options when C = 0.2 for Examples 
3, 4 and 5. Example 4 favours keeping Yj^ = 1 whereas other 
two take less number of iterations with Yj^ generated by (61). 
If in Step 6a of the algorithm in Section 3.5, j > 0 for a 
good number of iterations, it is preferred to keep s 1. 

When second order statistics of x^, which is modelled 
as random vector, are unknown, becomes a design criterion. 
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TABLE 1 Feedback and cost matrices with Example 1 for comparison 
with transfer function approach 

Fj = [0.9245 0.1705 0.0406] 

F* = [0.6229 0.1090 0.0241] 


fo .3934 0.03945 

0.00389 

0.01067 

0.03060 

0.03945 0.00574 

0.00062 

0.00185 

0.00518 

0.00389 0.00062 

0.00016 

0.00023 

0.00025 

0.01067 0.00185 

0.00023 

0.00080 

0.00234 

_0 .03060 0.00518 

0.00025 

0.00234 

0.01760 

0.35750 0.03457 

0.00270 

0.00787 

0.0332 T 

0.03457 0.00432 

0.00039 

0.00125 

0.00634 

0.00270 0.00039 

0.00004 

0.00011 

0.00037 

0.00787 0.00125 

0.00011 

0.00040 

0.00179 

0.03321 0.00634 

0.00037 

0.00179 

0.01347 


Tr [S^] = 0.41765 
Tr [S*] = 0.38636 

Optimal cost with state feedback, for E [x x' ] = I, 

0 0 “^ ^ 


= 0.37768 
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TABLE 2 Illu q t + T nrk 

lustration of a non-dominant optimal solution using 
Example 2 



0.0 

0.0 



~ 7.143 

0.000 

o.ooo" 

F = 







0 

0.0 

0.0 


S° = 

0.000 

5.000 

0.000 






_ 0.000 

0.000 

25.000_ 


0.0794 

0. 

3486 


8.187 

-1.417 

0.163~ 

F* = 

^0.1133 

1.: 

2660 

s* = 

-1.417 

2.825 

-0.616 




—i 


_0.163 

-0.616 

4.172 


""0.017 

0.180 


r 7.087 

-0.270 

-0.056~ 

> 

II 




Sa = 

-0.270 

5.557 

—0 . 660 


0.044 

1.340 





I 





_5).050 

—0.660 

4.105 


For 

Xq = [l 00], J® = 7.143 , J* = 8.187 


For 

Xq = [O 1 0] , J® = 5.000, = 5.557 
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TABLE 3 Comparison with modified Levine-Athans Algorithms 
for Example 3 



Dual 

Levine- 

-Athans Algm. 

Levine 

Algm. 

-Athans 

Modified 
DAM Algm, 


k 

J(Fj^) 

liAF^II 

J(F^) 

1 lAF^I 1 

J(F^) 

IIaf.I 

0 

2894.1 

170 

2894.1 

97 

2894.1 

98.57 

1 

1866.2 

2.2 

2194.4 

42 

1885.9 

37.97 

'2 

1764.4 

3.7 

1896.4 

15 

1754.6 

11.51 

3 

1745.3 

2.4 

1787.4 

4.9 

1740.1 

0.74 

4 

1740.3 

1.3 

1752.5 

1.6 

1738.05 

0.33 

5 

1738.7 

0.67 

1742.3 

0.55 

1737.94 

0.22 

6 

1738.3 

0.36 

1739.8 

0.21 

1737.93 

0.05 

7 

• 

1738.2 

0.19 

1738.5 

0.08 

1737.92 

0.04 

• 

• 

6 

1738.2 

_3 

<10 

1738.2 

< lO”^ 

1737.92 

< lO"^ 

nitial 

ptimal 

feedback gain matrix Fq = [-25 
output feedback gain matrix F* 

-285] 

= [-1.366 • 

-12.42] 
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TABLE 4 Comparison with modified Levine-Athans methods for 
Example 4 



Dual 

Levine-Athans Algm 

Levine 
. Algm, 

-At bans 

Modified 
DAM Algm. 


k 

J(F^) 


J(F,) 

1 Kf^ 1 1 

J(F,) 1 

lAF.II 

1 IdJ/dFl 1 

0 

31135 

10.0 

31135 

3600 

31135 

3537.4 

8.5 E+07 

1 

176.97 

1.1 

15590 

1800 

598.1 

21.3 

2186.82 

2 

162.22 

0.42 

7838.6 

900 

240.6 

4.85 

110.28 

3 

159.42 

0.23 

3981.3 

450 

189.0 

3.27 

41.56 

4 

159.34 

0.13 

2065.8 

220 

164.8 

2.1 

22.24 

5 

159.16 

0.075 

1109.9 

100 

160.2 

0.64 

6.72 . 

6 

159.10 

0.044 

630.4 

56 

159.35 

0.17 

2.31 

7 

# 

159 .08 

0.028 

389.0 

27 

159.19 

0.08 

1.33 

♦ 

• 

r 

159.07 

<10-^ 

159.07 

0.012 

159.068 

<10"^ 

0.01 


litial feedback gain matrix = 0 
timal output feedback gain matrix F* = 


-0.398 -1,592 -7.852 

1.257 3.480 5.004 


timal cost with state feedback - 151.246 
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TABLE 5 Comparison with 

steepest descent method for Example 4 


Steepest Des 
Method 


Cent 


Modified 
DAM Method 


J(F,) 


J(F^) 


0 

1 

2 

3 

4 

5 


31135.1 

8143.8 

2395.5 

1201.4 

900.9 

738.8 


31135.1 

589.1 
240.6 
189.0 
164.8 

160.2 


6 


629.8 


159.3 
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TABLE 6 Results for various C in Example 4 

9 


g 

No. of 
iterations 

No. of times 

MLE is solved 

CPU 

T ime 

(in sec.) 

0.5 

9 

44 

1.27 

0.4 

9 

43 

1.25 

0.3 

9 

43 

1.25 

0.2 

11 

34 

1.17 

p.l 

11 

34 

1.18 

0.04 

17 

76 

2.02 
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TABLE 7 Results for various in Example 5 


Ce, 

9 

JCFj) 

1 lAFj^l 1 

J(Fio) 

llAFioll No.of No.of 
iter, times 
MLE is 
solved 

CPU 
time 
(in sec. 

0.5 

135.54 

3.68 

133.7 

0.082 13 

72 

9.56 

0.3 

135.54 

3.68 

133.7 

0.082 13 

67 

9.05 

oa 

135.54 

3.68 

133.7 

0.040 13 

60 

8.32 

0.04 

134.27 

1.60 

133.7 

0.02 12 

55 

7.68 

0.02 

134.27 

1.60 

133.7 

0.02 12 

55 

7.68 

0.01 

196.27 

12.6 

134.2 

2.16 16 

69 

9.55 


8 Comparison of y 

= 1 and Yjf 1 using 

Examples 3 

,4 and 5 

Exampl# 

V istt 

Tic “ 

1 


generated 






using 

(61) in Chap. 3 


No.of No. 

of LME 

No. of 

No. of LME 


Ittr. calculations iter. 

calculations 


12 

76 

11 

34 

9 

39 

11 

46 

16 

79 

13 

62 


5 
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TABLE 9 Illustration of advantage of minimum norm feedback 
using Example 7 


F 



1*151 

0 

0 

0 


-0.56 

0 

0 

0 


0 

2.3 

0.82 

0 


0 

0.62 

3.41 

0 


0 

0 

0 

4.96 


0 

0 

0 

0.06 


Numbtr of cost function evaluations = 20 

Minimum norm feedback as = 

0 


1*222 

0 

0 

0 


0.469 

0 

0 

0 


0 

1.159 

0.588 

0 


0 

0.5165 

2.248 

0 


0 

0 

0 

1.168 


0 

0 

0 

0.322 


Number of cost function evaluations = 15 
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TABLE 10 Dynamic compensators for Example 5 

*^0? ’ '^c3 - 0- Q<,i = = I 

For n * 0, 

-2.601 -0.396 2.718 -0.053 

F» * -0.998 -2.412 A ciAn 

L 4.360 -3.740 

J* * 133.7 -* 

For n_ « 1, 

C 



"^2.193 

-0.584 

3.109 

-0.031 

I. 45 J 


P# a 

-0,934 

-2.458 

4.808 

-3.643 

0.642 



.418 

-0,204 

3.712 

0.749 

1.554 


J* - 129.624 






For n. a 
c 

■ 







’^2.592 

-0.3861 

2,712 

-0.038 

0.0554 0.014~] 


-0.9536 

-2.3890 

4.282 

-3.707 

0.0201 

0.004 

F* - 

-0.0395 

-0.0127 

0.0852 

0.008 

0.0556 

-0.0002 


-0.0052 

-0,0056 

0.0120 

-0.0014 

-0.0001 

0.5566 


J» • 136.068. 


74 


TABLE 11 Cost matrices for various X^' s in Example 8 

2 0 


F = 0 



^o" 


1 

0 


0 

0 


So = 


10 


1) I 

1 

0 


3.08 

-1.76 


X 

o 

11 

_p 

1 _ 

s* = 

_-1.76 

5.4q_ 



1> 

o“^ 


~3.08 

-1.7^ 


X 

o 

II 

0 

5 

s* = 

-1.76 

5.40 



Is 

0 


'“1.263 

-0.395] 

3 ) X.= 



s* = 




o 

0 

1 


—0 • 395 

9.94^ 


s* = 


1.236 -3.59E-04 

-3.59E-04 10.76 




CHAPTER 7 


CONCLUSION 


satis- 
in [28 J . 


The algorithm presented In this thesis can M usad « a 
general purpose algorithm for the design of optimal output 
feedback regulators. The descent Anderson Moo« method whicl 
has been coded in Fortran IV for creating CAD package perfor- 
med satisfactorily for a wide class of systeas. There are 
examples for which the method has failed to converge 
factorilyj one of them being the test example given 

The algorithm is written for systems with direct feed- 
through and for general quadratic performance indicee. As 
stated in Remark 3.3.6, the algorithm can be used for calcu- 
lating optimal output feedback gains for stochastic systems 
with plant disturbances assumed as white noise processes . 

When faced with convergence problems, penalty 
elements could help in convergence and uniqueness of the 
solution L35]. vwien the number of free parameters in the 
feedback gain matrix is large, constraining some of the 
elements helps in convergence of the algorithm. 

The efficiency of the algorithm depends greatly 
design parameters chosen. 


on feedbac 


on the 


APPENDIX 


In the system equations. 


X = Ax + Bu 

Y. ~ C )c + 

= CpX + DpU ; 

and index of performance 

oo ^ 

J =S[/ Q y, 

With E(^ Xj) = . 

A, B, C, D, Cp, Dp, Q, R and for different examples a! 
given by ; 

Example 1 



Example 2 
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A = 


~^.07 

0 

0 *“ 


0.1 ( J 1 

0 

“ 0.1 

0 

B = 

1 0 

_ 0 

0 

“0.4 


_ 0*2 1 


Cp = 1 


3x3 - D = D = 0, Q 


Example 3 


1 

0 


0 

1 


c = 


1 

0 




19 0 20 


R = 


0 1 0 


o' 


“o -1 “ 

O 

o 

B = 

0 

c = 

5 -1 

[0 13 ^ 


1 


-1 0 

— — i 


A = 


^ % 0, Q - R = 1, = I 

Example 4 

-0.037 0.0123 0.00055 -l“ 

° 0 1 0 
0 -0.23 0.0618 

^ 0.016 -0.0457 


3x3 


B= 


0.0008 
0 

0.08 
-0.0862 


0 1 0 
0 0 1 
0 0 0 


S = 

Q = 


^4x4’ D - Dp = 0, 

^4x4» ^ ~ ^2x2* 


= I 


1 0 
0.4 3 

6 

1 


^3x3 


0.000231 

0 

0.804' 

“0.0665 


4x4 
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Example 5 


-20 

0 

0 

-0.744 

0.337 

0.02 

0 


0 

-25 

0 

-0.032 

• 1.12 

0 

0 


o 0 

0 0 

0 0 

0 -0.154 

0 0.249 

0.0386 -0.996 

0 0.5 


-0.0042 1,54 

-1»0 -5.20 

-0.00029 -0. 117 
0 0 


0 

0 

0 

0 

0 

0 

-0.5 


B • = 


20 0 0 0 0 0 0 

-p 25 0 0 0 0 ^ 

° 0 0 1 0 0 -1 

° 0 0 0 1 0 

° 0 0 0 0 1 


0 10 0 0 
Q - diag. [1,1,30,30,5,5,1], 
Example 6 


’ S - hxT Dp = D 


R = diag. 


-0.154 0.004 0.99 0.178 0.0^ 

-1.25 -2.85 1.43 o -0.727 

0.568 -0.277 -0.284 O .7 ara 


1 

0 

1 0 
0 1 
0 0 


-0.727 

-2.050 

0 

-10 



0 


0 

B = 

0 


0 


1 


0 0 


^ 0 0 , C = D 

* P ■^5x5’ -p 


= D = 0 
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